The Use of kNN Model Algorithm for Automatic Feature Selection

An investigation has been conducted on frequently used feature selection methods in data pre-processing for data mining. After identifying the weaknesses and strengths of each of the approaches, we propose modifications to the ReliefF methods by: (1) using a kNN model as the starter selection, aimed at choosing a set of more meaningful representatives to replace the original data for feature selection; (2) integration of the Heterogeneous Value Difference Metric to handle heterogeneous applications – those with both ordinal and nominal features; and (3) presenting a simple method of difference function calculation. We have evaluated the performance of the proposed kNN model-based feature selection method on toxicity data for phenols using a linear regression algorithm. Experimental results indicate that the proposed feature selection method has a significant improvement in the classification accuracy for the trial data set.